ATUM SaaS - QA Test Document
**Document Version:** 1.0
**Last Updated:** 2026-03-06
**Platform:** ATUM SaaS - AI Workforce Platform
---
Test Case Legend
| Column | Description |
|---|---|
| **Test ID** | Unique test case identifier |
| **Feature** | Feature or component being tested |
| **Test Case** | Description of what is being tested |
| **Test Steps** | Step-by-step instructions to execute the test |
| **Expected Result** | Expected outcome if test passes |
| **Priority** | P0 (Critical), P1 (High), P2 (Medium), P3 (Low) |
| **Type** | Functional, Integration, E2E, Security, Performance |
---
Table of Contents
- Authentication & Authorization
- Agent Management
- Multi-Tenancy
- Canvas System
- Brain Systems
- Integrations
- Sales & CRM
- Security & Governance
- Package Management
- Billing & Pricing
- Admin & Monitoring
- Dashboard & Analytics
- Marketplace
- Settings & Configuration
---
1. Authentication & Authorization
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| AUTH-001 | User Signup | Verify new user can sign up with valid credentials | 1. Navigate to /auth/signup<br>2. Enter valid email, password, and confirm password<br>3. Click "Sign Up"<br>4. Check email for verification link | User account created, verification email sent, redirect to verification page | P0 | Functional |
| AUTH-002 | User Signup | Verify password validation during signup | 1. Navigate to /auth/signup<br>2. Enter email with weak password (less than 8 chars)<br>3. Click "Sign Up" | Error message displayed: "Password must be at least 8 characters" | P0 | Functional |
| AUTH-003 | User Signup | Verify email uniqueness validation | 1. Navigate to /auth/signup<br>2. Enter already registered email<br>3. Click "Sign Up" | Error message: "Email already registered" | P1 | Functional |
| AUTH-004 | User Login | Verify user can login with valid credentials | 1. Navigate to /auth/signin<br>2. Enter registered email and password<br>3. Click "Sign In" | User redirected to dashboard, session established | P0 | Functional |
| AUTH-005 | User Login | Verify login fails with invalid credentials | 1. Navigate to /auth/signin<br>2. Enter invalid email/password<br>3. Click "Sign In" | Error message: "Invalid credentials" | P1 | Functional |
| AUTH-006 | Email Verification | Verify email verification link works | 1. Click verification link from email<br>2. Wait for redirect | Email verified, user redirected to dashboard with success message | P0 | Functional |
| AUTH-007 | Password Reset | Verify password reset request | 1. Navigate to /auth/signin<br>2. Click "Forgot Password"<br>3. Enter registered email<br>4. Click "Send Reset Link" | Password reset email sent, confirmation message displayed | P1 | Functional |
| AUTH-008 | 2FA Setup | Verify 2FA can be enabled | 1. Navigate to /settings/security<br>2. Click "Enable 2FA"<br>3. Scan QR code with authenticator app<br>4. Enter verification code | 2FA enabled successfully, backup codes provided | P1 | Security |
| AUTH-009 | 2FA Login | Verify login requires 2FA code when enabled | 1. Login with email/password<br>2. Enter 2FA code from authenticator | Access granted after valid 2FA code | P1 | Security |
| AUTH-010 | SSO Login | Verify SSO login with Google OAuth | 1. Navigate to /auth/sso<br>2. Click "Sign in with Google"<br>3. Authorize with Google account | User logged in, account created/linked | P1 | Integration |
| AUTH-011 | Session Management | Verify session persists across page refreshes | 1. Login to application<br>2. Refresh page | User remains logged in, session data preserved | P1 | Functional |
| AUTH-012 | Session Expiry | Verify session expires after inactivity | 1. Login to application<br>2. Wait for session timeout (30 min)<br>3. Perform any action | Redirected to login page with "Session expired" message | P1 | Security |
| AUTH-013 | Logout | Verify user can logout successfully | 1. Click logout button<br>2. Confirm logout | Session cleared, redirected to login page | P0 | Functional |
| AUTH-014 | Admin Key Bypass | Verify admin key bypasses auth for testing | 1. Add X-Admin-Key header to request<br>2. Call protected endpoint | Access granted without standard auth | P1 | Security |
---
2. Agent Management
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| AGENT-001 | Agent Creation | Verify new agent can be created | 1. Navigate to /agents/new<br>2. Enter agent name, role, description<br>3. Select maturity level<br>4. Click "Create Agent" | Agent created, added to agent list | P0 | Functional |
| AGENT-002 | Agent Creation | Verify agent name uniqueness | 1. Navigate to /agents/new<br>2. Enter name of existing agent<br>3. Click "Create Agent" | Error: "Agent name already exists" | P1 | Functional |
| AGENT-003 | Agent Creation | Verify all maturity levels available | 1. Navigate to /agents/new<br>2. Check maturity level dropdown | Options: Student, Intern, Supervised, Autonomous | P1 | Functional |
| AGENT-004 | Agent Editing | Verify agent details can be edited | 1. Navigate to /agents<br>2. Click edit on existing agent<br>3. Update name, role, or description<br>4. Save changes | Changes saved and reflected in agent details | P1 | Functional |
| AGENT-005 | Agent Deletion | Verify agent can be deleted | 1. Navigate to /agents<br>2. Click delete on agent<br>3. Confirm deletion | Agent removed from list, associated data archived | P1 | Functional |
| AGENT-006 | Agent Execution | Verify student agent can perform read-only tasks | 1. Create agent with Student maturity<br>2. Assign search task<br>3. Execute task | Task completes with search results | P0 | Functional |
| AGENT-007 | Agent Execution | Verify student agent blocked from write operations | 1. Create agent with Student maturity<br>2. Assign task requiring data modification<br>3. Execute task | Error: "Insufficient permissions for this operation" | P0 | Security |
| AGENT-008 | Agent Execution | Verify intern agent requires approval for actions | 1. Create agent with Intern maturity<br>2. Assign action task<br>3. Execute task | Proposal created, approval required | P0 | Functional |
| AGENT-009 | Agent Execution | Verify supervised agent queues when unavailable | 1. Create agent with Supervised maturity<br>2. Set user availability to offline<br>3. Assign task | Task queued, notification sent when user available | P0 | Functional |
| AGENT-010 | Agent Execution | Verify autonomous agent can execute independently | 1. Create agent with Autonomous maturity<br>2. Assign complex task<br>3. Execute task | Task completes without human intervention | P0 | Functional |
| AGENT-011 | Agent Limits | Verify free tier limited to 3 agents | 1. Create 3 agents on free tier<br>2. Attempt to create 4th agent | Error: "Agent limit reached. Upgrade to create more agents." | P1 | Functional |
| AGENT-012 | Agent Limits | Verify solo tier allows 10 agents | 1. Upgrade to Solo tier<br>2. Create up to 10 agents | All 10 agents created successfully | P1 | Functional |
| AGENT-013 | Agent History | Verify agent execution history is tracked | 1. Execute agent task<br>2. Navigate to agent history<br>3. View execution log | Task logged with timestamp, inputs, outputs, status | P1 | Functional |
| AGENT-014 | Agent Terminal | Verify live agent terminal works | 1. Navigate to agent studio<br>2. Open agent terminal<br>3. Execute task | Real-time task output displayed in terminal | P1 | Functional |
| AGENT-015 | Agent Feedback | Verify feedback can be submitted for agent actions | 1. Execute agent task<br>2. Click "Provide Feedback"<br>3. Submit rating and comments | Feedback recorded, agent learning updated | P1 | Functional |
| AGENT-016 | Agent Governance | Verify governance rules enforced | 1. Create governance rule restricting action type<br>2. Attempt restricted action with agent | Action blocked by governance | P1 | Security |
---
3. Multi-Tenancy
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| MT-001 | Tenant Isolation | Verify tenants cannot access each other's data | 1. Login as tenant A user<br>2. Note tenant ID in URL<br>3. Attempt to access tenant B data via API | 403 Forbidden or 404 Not Found | P0 | Security |
| MT-002 | Tenant Isolation | Verify agents filtered by tenant_id | 1. Create agents in two different tenants<br>2. Query agents API | Each tenant only sees their own agents | P0 | Security |
| MT-003 | Subdomain Routing | Verify subdomain routing works correctly | 1. Navigate to tenant-a.atom-saas.fly.dev<br>2. Check tenant context | Correct tenant loaded from subdomain | P0 | Functional |
| MT-004 | RLS | Verify Row-Level Security enforced at DB level | 1. Direct database query with tenant A credentials<br>2. Query tenant B tables | Empty result set, RLS policy blocks access | P0 | Security |
| MT-005 | S3 Isolation | Verify S3 paths are tenant-specific | 1. Upload file for tenant A<br>2. Check S3 path | File stored in s3://atom-saas/{tenant_id}/ path | P0 | Security |
| MT-006 | Redis Isolation | Verify Redis keys are tenant-scoped | 1. Set Redis key for tenant A<br>2. Attempt to read from tenant B context | Key not found, proper namespacing (tenant:{id}:key) | P0 | Security |
| MT-007 | Tenant Creation | Verify new tenant can be created | 1. Submit tenant creation request<br>2. Set subdomain, admin email | Tenant created, unique subdomain assigned | P0 | Functional |
| MT-008 | Tenant Invitation | Verify users can be invited to tenant | 1. Navigate to tenant/invite<br>2. Enter user email<br>3. Select role<br>4. Send invitation | Invitation email sent, user added to tenant | P1 | Functional |
| MT-009 | Tenant Deletion | Verify tenant data is isolated on deletion | 1. Delete tenant A<br>2. Verify tenant B data intact | Only tenant A data removed, tenant B unaffected | P1 | Functional |
---
4. Canvas System
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| CV-001 | Canvas Browser | Verify browser canvas can navigate websites | 1. Navigate to /canvas/browser/{id}<br>2. Enter URL in address bar<br>3. Click "Go" | Website loads in canvas iframe | P0 | Functional |
| CV-002 | Canvas Browser | Verify browser canvas captures DOM | 1. Load webpage in browser canvas<br>2. Click "Capture DOM"<br>3. Check captured data | Page DOM structure captured and stored | P1 | Functional |
| CV-003 | Canvas Coding | Verify code execution canvas works | 1. Navigate to /canvas/coding/{id}<br>2. Write Python code<br>3. Click "Execute" | Code executes, output displayed | P0 | Functional |
| CV-004 | Canvas Coding | Verify code execution is isolated | 1. Execute code accessing filesystem<br>2. Check result | Access denied or isolated environment | P1 | Security |
| CV-005 | Canvas Email | Verify email canvas can read emails | 1. Navigate to /canvas/email/{id}<br>2. Connect email account<br>3. Load emails | Email list displayed in canvas | P1 | Functional |
| CV-006 | Canvas Email | Verify email canvas can compose emails | 1. Open email canvas<br>2. Click "Compose"<br>3. Enter recipient, subject, body<br>4. Click "Send" | Email sent successfully | P1 | Functional |
| CV-007 | Canvas Spreadsheet | Verify spreadsheet canvas can load CSV | 1. Navigate to /canvas/spreadsheet/{id}<br>2. Upload CSV file<br>3. Click "Load" | Data displayed in spreadsheet format | P1 | Functional |
| CV-008 | Canvas Spreadsheet | Verify spreadsheet can execute formulas | 1. Load data in spreadsheet canvas<br>2. Add formula cell (e.g., =SUM(A1:A10))<br>3. Check result | Formula calculated correctly | P1 | Functional |
| CV-009 | Canvas Terminal | Verify terminal canvas can execute commands | 1. Navigate to /canvas/terminal/{id}<br>2. Enter shell command<br>3. Press Enter | Command executes, output displayed | P1 | Functional |
| CV-010 | Canvas Terminal | Verify terminal is rate-limited | 1. Execute multiple commands rapidly<br>2. Check rate limit | Rate limit enforced after threshold | P1 | Security |
| CV-011 | Canvas Persistence | Verify canvas state is saved | 1. Open canvas, make changes<br>2. Close canvas<br>3. Reopen canvas | Changes restored from previous state | P1 | Functional |
| CV-012 | Canvas Sharing | Verify canvas can be shared with users | 1. Open canvas<br>2. Click "Share"<br>3. Add user email<br>4. Set permissions | User granted access to canvas | P1 | Functional |
| CV-013 | Canvas Audit | Verify canvas actions are audited | 1. Perform action in canvas<br>2. Check audit logs | Action logged with timestamp, user, canvas ID | P1 | Security |
| CV-014 | Canvas Integration | Verify canvas can integrate with skills | 1. Open canvas<br>2. Browse skill marketplace<br>3. Install skill<br>4. Use skill in canvas | Skill functionality available in canvas | P1 | Integration |
---
5. Brain Systems
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| BR-001 | Cognitive Architecture | Verify cognitive architecture initializes agent | 1. Create new agent<br>2. Call cognitive architecture initialize<br>3. Check agent state | Agent initialized with reasoning, memory, attention | P0 | Functional |
| BR-002 | Cognitive Architecture | Verify reasoning process generates decisions | 1. Initialize agent with cognitive architecture<br>2. Submit task<br>3. Call reasoning function | Decision returned with rationale | P1 | Functional |
| BR-003 | World Model | Verify experiences are recorded | 1. Execute agent task<br>2. Check world model storage | Experience stored with agent_id, task_type, outcome | P0 | Functional |
| BR-004 | World Model | Verify semantic search retrieves relevant experiences | 1. Record multiple experiences<br>2. Search by task description<br>3. Check results | Relevant experiences returned ranked by similarity | P1 | Functional |
| BR-005 | World Model | Verify world model respects tenant isolation | 1. Record experience for tenant A<br>2. Search from tenant B | No results returned, proper isolation | P0 | Security |
| BR-006 | Learning Engine | Verify patterns are learned from experiences | 1. Record multiple similar experiences<br>2. Generate adaptations<br>3. Check learned patterns | Patterns identified and adaptations suggested | P1 | Functional |
| BR-007 | Learning Engine | Verify RLHF improves agent performance | 1. Submit feedback on agent actions<br>2. Execute similar task<br>3. Check performance | Performance improves based on feedback | P1 | Functional |
| BR-008 | Reasoning Engine | Verify proactive interventions triggered | 1. Set reasoning rule for opportunity<br>2. Trigger condition<br>3. Check interventions | Intervention generated with URGENT/OPPORTUNITY/AUTOMATION type | P1 | Functional |
| BR-009 | Cross-System Reasoning | Verify data correlation across integrations | 1. Connect Salesforce and Slack integrations<br>2. Trigger cross-system reasoning<br>3. Check correlations | Correlated insights from both systems | P2 | Functional |
| BR-010 | Episodic Memory | Verify episodes capture context | 1. Execute agent task<br>2. Check episode record | Episode stored with task, context, outcome | P0 | Functional |
| BR-011 | Graduation System | Verify readiness score calculated correctly | 1. Execute agent tasks<br>2. Check readiness score | Score = zero_intervention_ratio*0.4 + constitutional_score*0.3 + confidence_score*0.2 + success_rate*0.1 | P0 | Functional |
| BR-012 | Graduation System | Verify student promotes to intern at 70% | 1. Achieve 70% readiness with student agent<br>2. Check maturity level | Agent promoted to intern | P0 | Functional |
| BR-013 | Graduation System | Verify intern promotes to supervised at 80% | 1. Achieve 80% readiness with intern agent<br>2. Check maturity level | Agent promoted to supervised | P0 | Functional |
| BR-014 | Graduation System | Verify supervised promotes to autonomous at 95% | 1. Achieve 95% readiness with supervised agent<br>2. Check maturity level | Agent promoted to autonomous | P0 | Functional |
| BR-015 | Episode Feedback | Verify feedback can be submitted for episodes | 1. Navigate to episode<br>2. Submit rating (0-1) and comments<br>3. Check feedback record | Feedback stored, used for RLHF | P1 | Functional |
---
6. Integrations
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| INT-001 | OAuth Flow | Verify OAuth authorization flow | 1. Navigate to integrations<br>2. Click "Connect" on any integration<br>3. Authorize on provider site<br>4. Wait for redirect | Integration connected, OAuth token stored | P0 | Integration |
| INT-002 | OAuth Flow | Verify OAuth token is refreshed | 1. Connect OAuth integration<br>2. Wait for token to approach expiry<br>3. Make API call | Token refreshed automatically, call succeeds | P1 | Integration |
| INT-003 | OAuth Flow | Verify OAuth can be disconnected | 1. Navigate to integrations<br>2. Click "Disconnect" on connected integration | Integration disconnected, tokens removed | P1 | Functional |
| INT-004 | Salesforce | Verify Salesforce can fetch contacts | 1. Connect Salesforce integration<br>2. Click "Fetch Contacts"<br>3. Check results | Contacts displayed from Salesforce | P1 | Integration |
| INT-005 | Salesforce | Verify Salesforce can create lead | 1. Connect Salesforce integration<br>2. Submit new lead data<br>3. Verify in Salesforce | Lead created in Salesforce | P1 | Integration |
| INT-006 | Slack | Verify Slack can send message | 1. Connect Slack integration<br>2. Compose message<br>3. Select channel<br>4. Send | Message posted to Slack channel | P1 | Integration |
| INT-007 | Slack | Verify Slack can read messages | 1. Connect Slack integration<br>2. Select channel<br>3. Click "Fetch Messages" | Recent messages displayed | P1 | Integration |
| INT-008 | Google Drive | Verify Google Drive can list files | 1. Connect Google Drive<br>2. Navigate to files section<br>3. Click "Refresh" | Files listed from Google Drive | P1 | Integration |
| INT-009 | Google Drive | Verify Google Drive can upload file | 1. Connect Google Drive<br>2. Select file to upload<br>3. Click "Upload" | File uploaded to Google Drive | P1 | Integration |
| INT-010 | HubSpot | Verify HubSpot can fetch deals | 1. Connect HubSpot integration<br>2. Navigate to deals<br>3. Click "Sync" | Deals imported from HubSpot | P1 | Integration |
| INT-011 | GitHub | Verify GitHub can fetch repositories | 1. Connect GitHub integration<br>2. Navigate to repos<br>3. Click "Load Repos" | Repositories listed from GitHub | P2 | Integration |
| INT-012 | Webhooks | Verify webhooks receive external events | 1. Configure webhook URL<br>2. Trigger event from external system<br>3. Check webhook log | Event received and logged | P1 | Integration |
| INT-013 | Integration Hub | Verify all 39+ integrations listed | 1. Navigate to /integrations<br>2. Check integration count | All available integrations displayed | P2 | Functional |
| INT-014 | Integration Health | Verify integration health status displayed | 1. Navigate to integrations<br>2. Check connection status | Status shows: Connected/Disconnected/Error | P1 | Functional |
---
7. Sales & CRM
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| SALES-001 | Lead Management | Verify lead can be created | 1. Navigate to sales section<br>2. Click "New Lead"<br>3. Enter lead details<br>4. Save | Lead created and added to list | P0 | Functional |
| SALES-002 | Lead Management | Verify lead can be updated | 1. Open existing lead<br>2. Update lead status or details<br>3. Save changes | Changes saved and reflected | P1 | Functional |
| SALES-003 | Lead Management | Verify lead can be deleted | 1. Open existing lead<br>2. Click "Delete"<br>3. Confirm | Lead removed from list | P1 | Functional |
| SALES-004 | Lead Scoring | Verify leads are automatically scored | 1. Create new lead<br>2. Wait for scoring process<br>3. Check lead score | Score assigned based on lead criteria | P1 | Functional |
| SALES-005 | Sales Intelligence | Verify insights are generated for leads | 1. Navigate to sales intelligence<br>2. Select lead<br>3. Check insights | Insights displayed based on lead data and integrations | P1 | Functional |
| SALES-006 | Sales Dashboard | Verify dashboard displays key metrics | 1. Navigate to sales dashboard<br>2. Check metrics | Total leads, conversion rate, revenue shown | P1 | Functional |
| SALES-007 | Sales Dashboard | Verify dashboard filters work | 1. Apply date range filter<br>2. Apply status filter<br>3. Check data | Dashboard updates with filtered data | P1 | Functional |
| SALES-008 | Sales Automation | Verify agent can automate sales tasks | 1. Create sales automation rule<br>2. Trigger condition<br>3. Check agent action | Agent executes automated sales task | P1 | Functional |
---
8. Security & Governance
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| SEC-001 | Rate Limiting | Verify rate limit enforced on API | 1. Make rapid requests to API endpoint<br>2. Check response after threshold | 429 Too Many Requests returned | P0 | Security |
| SEC-002 | Rate Limiting | Verify rate limit is per-tenant | 1. Make requests from tenant A<br>2. Make requests from tenant B<br>3. Check limits | Each tenant has independent rate limit | P0 | Security |
| SEC-003 | Agent Governance | Verify governance rules block unauthorized actions | 1. Create governance rule blocking data deletion<br>2. Attempt deletion with agent | Action blocked, rule violation logged | P0 | Security |
| SEC-004 | Agent Governance | Verify governance logs are auditable | 1. Check governance audit logs<br>2. Filter by agent, action, date | All governance decisions logged with timestamps | P1 | Security |
| SEC-005 | BYOK | Verify tenant can provide own API key | 1. Navigate to /settings/security<br>2. Enter OpenAI API key<br>3. Save | Key encrypted and stored in tenant_settings | P1 | Security |
| SEC-006 | BYOK | Verify tenant API key is used for embeddings | 1. Set tenant API key<br>2. Perform action requiring embeddings<br>3. Check API usage | Tenant's key used, not platform key | P1 | Security |
| SEC-007 | Audit Logging | Verify all user actions are logged | 1. Perform any action (create agent, etc.)<br>2. Check audit logs | Action logged with user, timestamp, details | P0 | Security |
| SEC-008 | Audit Logging | Verify audit logs are tamper-proof | 1. Attempt to modify audit log entry<br>2. Check result | Modification denied or logged | P1 | Security |
| SEC-009 | RBAC | Verify role-based access control works | 1. Create user with Viewer role<br>2. Attempt admin action | Access denied, 403 Forbidden | P0 | Security |
| SEC-010 | RBAC | Verify permissions can be customized | 1. Create custom role<br>2. Assign specific permissions<br>3. Assign to user<br>4. Test access | User has exactly assigned permissions | P1 | Functional |
| SEC-011 | Data Encryption | Verify sensitive data encrypted at rest | 1. Check database for API keys<br>2. Check S3 for sensitive files | Data encrypted, not plaintext | P0 | Security |
| SEC-012 | Data Encryption | Verify data encrypted in transit | 1. Check API connection<br>2. Check database connection | TLS/SSL enforced | P0 | Security |
---
9. Package Management
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| PKG-001 | Python Whitelist | Verify approved package can be installed | 1. Request installation of whitelisted package<br>2. Check installation | Package installed successfully | P1 | Functional |
| PKG-002 | Python Whitelist | Verify non-whitelisted package blocked | 1. Request installation of non-whitelisted package<br>2. Check result | Installation blocked, approval requested | P0 | Security |
| PKG-003 | Python Whitelist | Verify vulnerability scan runs on install | 1. Install whitelisted package<br>2. Check scan results | pip-audit or Snyk scan executed | P1 | Security |
| PKG-004 | Python Whitelist | Verify vulnerable package version blocked | 1. Attempt to install package with known vulnerability<br>2. Check result | Installation blocked, vulnerability reported | P0 | Security |
| PKG-005 | NPM Integration | Verify NPM package can be installed | 1. Request NPM package installation<br>2. Check result | Package installed in Docker container | P1 | Functional |
| PKG-006 | NPM Integration | Verify NPM audit runs on install | 1. Install NPM package<br>2. Check audit results | npm audit executed, vulnerabilities reported | P1 | Security |
| PKG-007 | Package Isolation | Verify packages isolated per skill | 1. Install package for skill A<br>2. Check skill B environment | Package not available to skill B | P1 | Security |
| PKG-008 | Package Approval | Verify admin can approve package | 1. Navigate to package whitelist<br>2. Submit package for approval<br>3. Admin approves<br>4. Install package | Package added to whitelist, installation succeeds | P1 | Functional |
| PKG-009 | postinstall Protection | Verify malicious postinstall blocked | 1. Attempt package with suspicious postinstall<br>2. Check execution | postinstall executed in isolated container | P0 | Security |
---
10. Billing & Pricing
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| BILL-001 | Pricing Tiers | Verify all 4 tiers available | 1. Navigate to /settings/billing<br>2. Check upgrade options | Free, Solo ($19), Team ($79), Enterprise ($299) shown | P1 | Functional |
| BILL-002 | Usage Tracking | Verify agent usage tracked | 1. Execute agent tasks<br>2. Check usage dashboard | Task count incremented, quota updated | P0 | Functional |
| BILL-003 | Usage Limits | Verify free tier limit of 50/day enforced | 1. Execute 50 agent tasks on free tier<br>2. Attempt 51st task | Error: "Daily limit reached" | P0 | Functional |
| BILL-004 | Usage Limits | Verify solo tier limit of 500/day | 1. Upgrade to Solo tier<br>2. Execute 500 agent tasks<br>3. Attempt 501st task | Error: "Daily limit reached" | P1 | Functional |
| BILL-005 | Billing Cycle | Verify billing cycle is monthly | 1. Check billing start date<br>2. Wait one month<br>3. Check invoice | Invoice generated for monthly period | P1 | Functional |
| BILL-006 | Invoice Generation | Verify invoice includes line items | 1. Navigate to billing<br>2. View latest invoice | Invoice shows: Plan, usage, total amount | P1 | Functional |
| BILL-007 | Payment Processing | Verify payment method can be added | 1. Navigate to billing<br>2. Click "Add Payment Method"<br>3. Enter card details<br>4. Save | Payment method added and verified | P0 | Functional |
| BILL-008 | Payment Processing | Verify payment fails with invalid card | 1. Attempt to add invalid card<br>2. Check result | Error: "Invalid payment method" | P1 | Functional |
| BILL-009 | Subscription Upgrade | Verify upgrade takes effect immediately | 1. Upgrade from Free to Solo<br>2. Check agent limit | Limit increased to 10 agents immediately | P1 | Functional |
| BILL-010 | Subscription Downgrade | Verify downgrade takes effect next cycle | 1. Downgrade from Team to Solo<br>2. Check current limits | Full Team features until cycle end | P1 | Functional |
| BILL-011 | Enterprise Plan | Verify enterprise allows unlimited agents | 1. Subscribe to Enterprise<br>2. Create agents beyond 25 | No agent limit enforced | P1 | Functional |
---
11. Admin & Monitoring
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| ADM-001 | Admin Dashboard | Verify admin dashboard accessible to admins | 1. Login as admin user<br>2. Navigate to /admin | Dashboard loads with admin controls | P0 | Functional |
| ADM-002 | Admin Dashboard | Verify non-admin cannot access admin dashboard | 1. Login as regular user<br>2. Navigate to /admin | 403 Forbidden or redirect | P0 | Security |
| ADM-003 | User Management | Verify admin can view all users | 1. Navigate to admin/users<br>2. Check user list | All users across tenants displayed | P1 | Functional |
| ADM-004 | User Management | Verify admin can deactivate user | 1. Select user<br>2. Click "Deactivate"<br>3. Confirm | User deactivated, cannot login | P1 | Functional |
| ADM-005 | Tenant Management | Verify admin can view all tenants | 1. Navigate to admin/tenants<br>2. Check tenant list | All tenants displayed with metrics | P1 | Functional |
| ADM-006 | Tenant Management | Verify admin can suspend tenant | 1. Select tenant<br>2. Click "Suspend"<br>3. Confirm | Tenant suspended, all access blocked | P1 | Functional |
| ADM-007 | System Health | Verify health metrics displayed | 1. Navigate to admin/health<br>2. Check metrics | CPU, memory, database, Redis status shown | P1 | Functional |
| ADM-008 | System Logs | Verify system logs accessible | 1. Navigate to admin/logs<br>2. Apply filters | Logs displayed with filtering options | P1 | Functional |
| ADM-009 | Governance Panel | Verify governance rules manageable | 1. Navigate to governance panel<br>2. Create/edit/delete rule | Rules CRUD operations work | P1 | Functional |
| ADM-010 | Interventions Feed | Verify interventions displayed in real-time | 1. Trigger agent intervention<br>2. Check interventions feed | Intervention appears with details | P1 | Functional |
---
12. Dashboard & Analytics
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| DASH-001 | Main Dashboard | Verify dashboard loads for authenticated user | 1. Login to application<br>2. Navigate to /dashboard | Dashboard displays with widgets | P0 | Functional |
| DASH-002 | Main Dashboard | Verify dashboard shows agent summary | 1. Check dashboard agents widget | Total agents, active agents, by maturity shown | P1 | Functional |
| DASH-003 | Industry Dashboard | Verify industry-specific dashboards load | 1. Navigate to /dashboard/marketing<br>2. Check content | Marketing-specific metrics and tools shown | P1 | Functional |
| DASH-004 | Analytics Dashboard | Verify analytics page displays charts | 1. Navigate to /analytics<br>2. Check visualizations | Charts for agent performance, usage, etc. | P1 | Functional |
| DASH-005 | Analytics Dashboard | Verify date range filter works | 1. Select custom date range<br>2. Apply filter | Charts update for selected period | P1 | Functional |
| DASH-006 | Analytics Dashboard | Verify data can be exported | 1. Click "Export"<br>2. Select format (CSV/JSON)<br>3. Download | File downloaded with correct data | P1 | Functional |
---
13. Marketplace
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| MKT-001 | Skill Marketplace | Verify marketplace displays skills | 1. Navigate to /marketplace<br>2. Browse skills | Skills listed with descriptions, ratings | P1 | Functional |
| MKT-002 | Skill Marketplace | Verify skills can be filtered | 1. Apply filters (category, rating, price)<br>2. Check results | Filtered skills displayed | P1 | Functional |
| MKT-003 | Skill Purchase | Verify free skill can be installed | 1. Select free skill<br>2. Click "Install"<br>3. Confirm | Skill added to tenant, ready to use | P1 | Functional |
| MKT-004 | Skill Purchase | Verify paid skill requires payment | 1. Select paid skill<br>2. Click "Purchase"<br>3. Check result | Payment flow initiated | P1 | Functional |
| MKT-005 | Skill Submission | Verify custom skill can be submitted | 1. Navigate to /canvas-skills/marketplace/submit<br>2. Upload skill<br>3. Enter details<br>4. Submit | Skill submitted for review | P1 | Functional |
| MKT-006 | Skill Rating | Verify skills can be rated | 1. Use installed skill<br>2. Submit rating and review<br>3. Check marketplace | Rating updated, review displayed | P1 | Functional |
| MKT-007 | Canvas Marketplace | Verify canvas components can be shared | 1. Create canvas component<br>2. Submit to marketplace<br>3. Check listing | Component appears in marketplace | P2 | Functional |
---
14. Settings & Configuration
| Test ID | Feature | Test Case | Test Steps | Expected Result | Priority | Type |
|---|---|---|---|---|---|---|
| SET-001 | Account Settings | Verify user profile can be updated | 1. Navigate to /settings/account<br>2. Update name, email<br>3. Save | Changes saved and reflected | P1 | Functional |
| SET-002 | Account Settings | Verify password can be changed | 1. Navigate to /settings/account<br>2. Enter current password<br>3. Enter new password<br>4. Save | Password updated, must re-login | P1 | Functional |
| SET-003 | Notification Settings | Verify notification preferences work | 1. Navigate to /settings/notifications<br>2. Toggle notification types<br>3. Save | Preferences saved, notifications updated | P1 | Functional |
| SET-004 | Security Settings | Verify 2FA can be enabled/disabled | 1. Navigate to /settings/security<br>2. Toggle 2FA<br>3. Follow setup/disable steps | 2FA status updated | P1 | Security |
| SET-005 | Governance Settings | Verify governance rules can be configured | 1. Navigate to /settings/governance<br>2. Create rule<br>3. Set conditions<br>4. Save | Rule created and enforced | P1 | Functional |
| SET-006 | SSO Configuration | Verify SSO can be configured | 1. Navigate to /settings/sso<br>2. Enter SSO provider details<br>3. Save | SSO configuration saved | P1 | Integration |
| SET-007 | Role Management | Verify custom roles can be created | 1. Navigate to /settings/roles<br>2. Click "New Role"<br>3. Set permissions<br>4. Save | Role created, can assign to users | P1 | Functional |
---
Test Execution Summary
Priority Breakdown
- **P0 (Critical):** 64 tests - Must pass for release
- **P1 (High):** 119 tests - Important features
- **P2 (Medium):** 11 tests - Nice to have
- **P3 (Low):** 0 tests - Future enhancements
**Total Test Cases:** 194
Type Breakdown
- **Functional:** 120 tests
- **Security:** 43 tests
- **Integration:** 24 tests
- **Performance:** 0 tests
- **E2E:** 7 tests
---
Test Coverage Areas
✅ **Covered:**
- Authentication & Authorization (14 tests)
- Agent Management (16 tests)
- Multi-Tenancy (9 tests)
- Canvas System (14 tests)
- Brain Systems (15 tests)
- Integrations (14 tests)
- Sales & CRM (8 tests)
- Security & Governance (12 tests)
- Package Management (9 tests)
- Billing & Pricing (11 tests)
- Admin & Monitoring (10 tests)
- Dashboard & Analytics (6 tests)
- Marketplace (7 tests)
- Settings & Configuration (7 tests)
---
Notes for Testers
- **Tenant Context:** Always verify tenant_id is properly filtered in all queries
- **Governance Checks:** Verify agent maturity levels are enforced for all actions
- **Rate Limits:** Check Redis-based rate limiting on all public endpoints
- **Brain Systems:** Test cognitive architecture, world model, and learning engine with real scenarios
- **Security:** Verify RLS, BYOK, and audit logging on all sensitive operations
- **Integration Testing:** Use test accounts for OAuth integrations
- **E2E Testing:** Run Playwright tests for critical user flows
---
Test Environment Setup
Required Test Data
- At least 3 different tenant accounts
- Admin user and regular users per tenant
- OAuth test accounts (Google, Slack, Salesforce)
- Test credit card for billing tests
- Sample files for upload testing
Test Accounts
- **Admin:** admin@atom-saas.com / AdminPassword123!
- **Tenant A User:** tenanta@test.com / TestPassword123!
- **Tenant B User:** tenantb@test.com / TestPassword123!
Test URLs
- **Production:** https://atom-saas.fly.dev
- **Staging:** [Configure if available]
- **Local:** http://localhost:3000
---
**Document Status:** ✅ Complete
**Ready for Import to Excel:** Yes - All tables use standard markdown format
**Last Review:** 2026-03-06